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EVALUATION  OF  THE  PROTOTYPE  POSITION  DATA  ANALYSIS  JOB  AID 
(PDAT-JA) 

EXECUTIVE  SUMMARY _ 


Rsquirenent : 

The  Position  Data  Analysis  Job  Aid  (PDAT-JA)  software  is  a 
product  of  research  accomplished  during  earlier  phases  of  the 
Military  Occupational  Specialty  (MOS)  restructuring  research  and 
development  program  conducted  by  the  U.S.  Army  Research  Institute 
for  the  Behavioral  and  Social  Sciences  (ARI) .  The  software, 
released  for  demonstration  in  December  1990,  automates  much  of 
the  position  data  analysis  process  performed  during  MOS  restruc¬ 
turing.  The  research  requirement  was  to  conduct  a  user  evalua¬ 
tion  of  the  prototype  PDAT-JA  to  identify  benefits  or  problems 
associated  with  its  use,  as  well  as  any  needed  or  desirable 
changes  to  either  the  software  or  the  user's  manual. 


Procedure : 

The  work  underlying  this  effort  involved  three  steps. 

First,  evaluation  procedures  were  developed  to  collect  opera¬ 
tional  data  on  the  PDAT-JA  software.  Next,  PDAT-JA  was  installed 
at  the  evaluation  sites,  and  software  evaluators  were  identified 
and  briefed.  Finally,  evaluation  data  were  collected  and 
analyzed,  and  results,  conclusions,  and  recommendations  were 
documented.  The  primary  focus  of  the  evaluation  was  to  identify 
improvements  that  PDAT-JA  contributed  to  the  position  data  analy¬ 
sis  process  and  to  obtain  evaluator  recommendations  for  software 
modification. 


Findings: 

This  research  note  identifies  two  primary  findings  concern¬ 
ing  the  prototype  PDAT-JA  software: 


PDAT-JA  enhances  the  MOS  analyst's  ability  to 
perform  position  data  analysis  by  substantially 
Improving  accuracy  and  reducing  the  time  require 
for  analysis  by  as  much  as  72  to  95  percent. 

Modifications  to  PDAT-JA  are  needed  to  augment 
the  software's  capability  to  assist  the  MOS 
analyst  in  performing  position  data  analysis. 
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utilization  of  Findings: 


The  findings  indicate  that,  with  further  modification,  PDAT- 
JA  can  significantly  reduce  the  time  required  to  perform  position 
data  analysis.  A  selected  subset  of  these  modifications  will  be 
made  and  a  revised  user's  manual  will  be  prepared.  The  new  PDAT- 
JA  software  and  user's  manual  will  be  distributed  to  the  person¬ 
nel  proponent  offices  in  the  Army. 
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EVALUATION  OF  THE  PROTOTYPE  POSITION  DATA  ANALYSIS  JOB  AID 

(PDAT-JA) 


Introduction 

This  report  documents  an  evaluation  of  the  prototype 
Position  Data  Analysis  Job  Aid  (PDAT-JA)  based  on  its  initial  use 
during  May-June  1991.  The  report  provides  a  brief  description  of 
the  software,  discusses  the  role  of  PDAT-JA  in  the  Military 
Occupational  Specialty  (MOS)  restructuring  research  and 
development  program,  documents  results  of  the  software's  test  and 
evaluation,  and  provides  recommended  software  modifications. 

The  purpose  of  this  report  is  to  document  the  extent  to 
which  PDAT-JA  supports  the  personnel  proponent  analyst  in 
performing  position  data  analysis.  The  primary  focus  of  the 
report  is  to  examine  the  "value  added"  to  MOS  restructuring  by 
the  prototype  software. 

Background 

The  PDAT-JA  software  is  a  product  of  research  conducted 
during  earlier  phases  of  the  Army  Research  Institute's  (ARI)  MOS 
restructuring  research  and  development  program  Akman  &  Haught 
(1990)  identified  the  position  data  analysis  phase  of  MOS 
restructuring  as  one  of  the  more  critical  and  demanding  steps  of 
operations-based  MOS  restructuring. 

During  follow-on  research  work,  specifications  for  PDAT-JA 
were  developed  to  provide  a  description  of  personal  computer 
(PC) -based  analytical  aids  that  could  be  used  to  support  position 
data  analysis  (Haught  &  Akman,  1990) .  This  effort  resulted  in  a 
demonstration  that  a  PC-based  analytical  aid  could  be  developed 
that  would  improve  the  overall  effectiveness  of  the  Army's 
personnel  proponent  agencies. 

Based  on  the  specifications,  prototype  software  for  PDAT-JA 
was  developed  for  ARI  demonstrating  the  technical  and  operational 
feasibility  of  developing  software  capable  of  supporting  the  more 
difficult  segments  of  MOS  restructuring  performed  by  the 
personnel  proponent  (Haught,  Schniebolk,  &  Akman,  1990) .  The 
software,  released  for  demonstration  in  December  1990,  automates 
much  of  the  position  data  analysis  process. 

Overview  of  the  Report 

This  report  consists  of  five  sections.  The  first  section 
provides  a  description  of  position  data  analysis  and  defines  the 
role  of  PDAT-JA  in  the  MOS  restructuring  research  and  development 
program . 
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The  second  section  describes  the  procedure  used  in 
evaluating  PDAT-JA.  This  section  Identifies  the  standards  and 
constraints  that  Influenced  the  design  of  the  software.  The 
methodology  used  to  evaluate  the  prototype  software  is  also 
described. 

The  third  section  presents  the  results  of  the  PDAT-JA 
software  evaluation.  The  section  presents  a  comparison  of 
position  data  analysis  performed  in  the  manual  mode  and  the 
performance  Improvements  in  each  step  of  analysis  resulting  from 
the  Introduction  of  PDAT-JA.  Also  provided  in  the  section  is  a 
summary  of  overall  time  savings  for  each  evaluation  subject  based 
on  comparing  analysis  performed  with  PDAT-JA  and  that  analysis 
performed  manually. 

In  the  fourth  section,  evaluator-recommended  modifications 
to  the  PDAT-JA  software  are  reviewed  and  analyzed.  From  analysis 
of  evaluator  recommendations,  a  final  list  of  recommended 
software  modifications  is  identified. 

The  final  section  discusses  feasible  future  evaluation  and 
development  efforts  for  the  PDAT-JA  software.  This  section 
presents  a  case  for  continued  evaluation  and  full-scale 
development  of  PDAT-JA. 
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POAT-JA  and  M08  Rastruoturlng 


The  purpose  of  this  section  is  to  provide  a  synopsis  of  the 
development  of  PDAT-JA  and  define  the  role  the  software  plays  in 
the  MOS  restructuring  research  and  development  program.  The 
section  also  defines  the  scope  of  the  position  data  analysis 
phase  of  NOS  restructuring  and  furnishes  a  description  of  the 
PDAT-JA  software. 

overview  of  Position  Data  Analysis 

The  position  data  analysis  phase  of  MOS  restructuring  is  a 
decailed  analysis  of  the  authorized  MOS  positions  that  are 
affected  by  changes  in  doctrine,  policy,  organizations,  or  the 
introduction  of  new  or  modified  equipment  systems.  The  results 
of  this  analysis  provide  the  proponent  with  a  broad  overview  of 
the  relative  health  of  the  MOS,  types  and  geographic  locations 
and  organizations  where  the  MOS  is  authorized,  total  authorized 
positions,  space  Imbalanced  MOS  (SIMOS)  implications,  grade 
structure  needs,  and  combat  probability  of  the  MOS. 

Position  data  analysis  requires  manipulating  and  analyzing 
large  volumes  of  MOS  data.  Currently,  most  personnel  proponent 
agencies  are  performing  position  data  analysis  in  a  "stubby 
pencil”  mode  consisting  of  (1)  manual  sorting  of  The  Army 
Authorization  Documents  System  (TAADS)  MOS  data,  (2)  performing 
"tick  mark”  counts  of  MOS  data  based  on  several  different 
analysis  criteria,  (3)  manually  applying  standards  of  grade  to 
the  MOS  data  by  marking  up  hard  copy  TAADS  documents,  (4) 
developing  reports  based  on  the  results  of  counts,  and  (5) 
manually  applying  the  average  grade  distribution  matrix  (AGM)  to 
NOS  data. 

Typically,  each  step  of  position  data  analysis  may  be 
performed  as  few  as  two  times  (once  on  current  TAADS  data  and 
once  on  modified  data)  and  as  many  as  six  times  during  the 
process  of  an  MOS  restructuring  action.  Because  position  data 
analysis  is  so  tedious  and  time  consuming,  this  phase  of  MOS 
restructuring  can  take  two  to  six  months  to  perform,  depending  on 
the  size  of  the  MOS  being  analyzed. 

The  Role  of  PDAT-JA  in  the  MOS  Restructuring  Process 

MOS  restructuring  occurs  in  two  phases:  requirements-based 
and  operations-based.  In  the  requiremei.ts-based  phase, 
restructured  MOSs  are  developed  in  response  to  various  triggering 
actions  such  as  the  acquisition  of  new  equipment  or  doctrinal 
changes,  among  other  possibilities.  If  approved,  a  series  of 
analyses  is  required  in  order  to  implement  the  MOS  changes  into 
the  Army's  personnel  system;  this  constitutes  the  operations- 
bused  phase  of  restructuring.  Position  data  analysis  is  one  of 
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the  operations-based  analytical  steps,  albeit  very  time  consuming 
and  difficult  as  currently  manually  performed. 

The  Military  Occupational  Specialty  fMOS)  Restructuring 
Research  and  Development  Blueprint  (Steinbach,  Akman,  &  Haught, 
1990)  identified  development  of  a  position  data  analysis  tool  as 
an  important  step  in  Improving  the  operations-based  MOS 
restructuring  process.  The  rationale  was  that  such  a  tool  would 
greatly  reduce  the  time  and  resources  required  for  position  data 
analysis  while  significantly  Improving  its  accuracy.  The 
importance  of  such  a  tool  is  founded  in  critical  requirements  for 
position  data  in  subsequent  steps  in  operations-based 
restructuring  including  personnel  data  analysis,  recruiting 
analysis,  and  pre-  and  post-standards  of  grade  authorization 
(SGA)  development  and  analysis. 

PDAT-JA  was  designed  to  improve  position  data  analysis  by: 

1.  Significantly  reducing  the  time  and  manpower 
required; 

2.  Increasing  data  accuracy  and  validity; 

3.  Providing  the  ability  to  more  efficiently  manage 
the  data  to  be  analyzed  in  an  MOS  restructuring 
action;  and 

4.  Providing  an  audit  trail  capability. 

In  sum,  PDAT-JA  represents  a  significant  enhancement  in  the 
analytical  tools  which  are  available  to  support  the  analysis 
required  to  implement  MOS  restructurings.  It  offers  a  more 
structured,  verifiable  means  for  conducting  position  data 
analysis. 

Software  Pesoriptlon 

PDAT-JA  is  a  PC-based  prototype  software  system  designed  to 
improve  operations-based  MOS  restructuring  by  reducing  the 
burdens  of  manually  analyzing  the  large  volumes  of  data  involved 
in  position  data  analysis.  The  prototype  software  was  developed 
specifically  to  support  the  analysis  process  by  utilizing  the 
capabilities  of  the  Army  standard  PC  system  to  provide  the  MOS 
analyst  with  automated  sorting  of  TAADS  data  and  supplying 
information  on  MOS  environmental,  structural,  and  deployment 
data.  PDAT-JA  is  also  designed  to  provide  the  analyst  with  the 
capability  to  make  revisions  to  TAADS  through  computer  generated 
worksheets,  generate  reports,  and  make  comparisons  between 
original  TAADS  data  and  data  modified  by  the  analyst. 

The  PDAT-JA  software  was  developed  using  an  off-the-shelf 
"shell"  software.  The  software  is  distributed  in  a  run-time 
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version  of  the  FOXPro  data  base  manageoient  system  and  dGE  graphic 
display  system.  At  the  current  stage  of  development,  PDAT-JA  can 
support  TAADS  data  base  downloading,  analysis,  and  revision  of  up 
to  three  enlisted  MOSs.  Presently,  the  software  will  accept 
TAADS  data  for  active  Army  components  only.  Based  on  the  results 
of  test  and  evaluation  and  the  overall  success  of  the  prototype 
software,  a  fully  operational  software  system  can  be  developed 
with  the  capacity  to  support  analysis  of  officer,  warrant 
officer,  civilian,  and  reserve  component  data  as  well. 


5 


PDAT-JA  System  Bvalustion  Procsdure 


The  objective  of  the  prototype  system  evaluation  is  to 
assess  how  well  PDAT-JA  supports  the  personnel  proponent  analyst 
in  performing  position  data  analysis.  Three  areas  are  considered 
in  this  evaluation: 

1.  Assessing  performance  improvement  in  position 
data  analysis  resulting  from  the  introduction  of 
PDAT-JA; 

2.  Assessing  the  user-system  interface  and  software 
acceptance;  and 

3.  Assessing  the  adequacy  of  the  PDAT-JA  User's 
Manual. 

The  primary  focus  of  this  evaluation  is  to  assess  the  "value 
added"  by  automating  portions  of  position  data  analysis  through 
the  PDAT-JA  software.  Emphasis  is  also  placed  on  the 
Identification  of  Improvements  that  can  be  made  by  modifying  the 
current  prototype  as  opposed  to  those  that  would  best  be 
accomplished  by  undertaking  additional  software  development.  The 
evaluation  of  PDAT-JA  will  identify  those  modifications  that 
maximize  the  quality  and  capability  of  the  existing  software. 

Since  PDAT-JA  was  developed  as  a  research  product,  a  number 
of  design  criteria  and  constraints  were  defined  as  a  guide  for 
developing  the  prototype  software.  The  criteria  provide  a  basis 
for  the  data  manipulation  and  data  reporting  capability  of  the 
software  and  summarize  the  principal  features  of  position  data 
analysis.  The  constraints  identify  the  operating  environment  for 
PDAT-JA.  The  evaluation  procedure  is  designed  to  evaluate  PDAT- 
JA  against  these  criteria  and  constraints. 

Criteria 

The  following  are  a  list  of  the  primary  design  criteria: 

1.  The  software  must  provide  the  capability  to  sort 
TAADS  data  and  provide  MOS  suimiiary  report?;  based 
upon  predefined  position  data  criteria; 

2.  The  software  must  provide  the  capability  to 
manipulate  TAADS  deta  and  develop  additional 
data  for  assessment  of  the  environment, 
structure,  and  deployment  of  MOSs  in  support  of 
position  data  analysis; 

3.  The  software  must  provide  the  capability  to 
generate  modified  MOS  data  based  on  user  input 
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and  compare  the  original  MOS  data  with  the 
modified  data;  and 

4.  The  software  must  assist  the  MOS  analyst  in 
performing  grade  structure  analysis  by 
supporting  the  assessment  of  career  progression 
and  stability  of  current  and  proposed  MOS 
structures . 

Constraints 

The  following  is  a  list  of  the  software  constraints: 

1.  The  software  must  operate  on  Army  standard  PC 
equipment,  e.g..  Zenith  Z-248  PC  with  one  5-1/4” 
floppy  disk  drive,  one  20  megabyte  hard  drive, 

CGA  or  VGA  graphics  adapter,  and  the  Alps  2000 
dot  matrix  printer  (or  equivalent) ; 

2.  The  software  must  be  exportable  and 
nonproprietary,  not  requiring  the  government  to 
purchase  additional  software  packages,  and  must 
operate  on  MS-DOS  computer  hardware  as  described 
above ; 

3.  Because  PDAT-JA  was  constrained  to  operating  on 
the  Army  standard  system,  the  software's  data 
import  and  data  manipulation  capability  was 
deliberately  limited  to  three  MOS  with  a  total 
of  5000  or  fewer  positions. 

The  limitation  of  the  number  of  positions  that  can  be 
analyzed  is  a  function  of  the  storage  capacities  of  usually 
available  computer  ec[uipment  and  not  the  PDAT-JA  program. 
PDAT-JA  itself  can  handle  any  number  of  positions.  Work  is 
currently  being  performed  to  allow  for  the  increase  in  capacity 
of  currently  generally  available  equipment. 

Method 

To  obtain  a  reliable  evaluation  of  the  PDAT-JA  software 
requires  that  the  interaction  of  all  the  system  components 
(human,  hardware,  and  software)  be  assessed  in  an  operational 
environment  and  that  data  be  obtained  during  actual  systems 
operation.  To  satisfy  these  needs,  the  method  described  below 
addresses  the  following  three  critical  areas: 

1.  Software  Installation; 

2.  Selection  of  Participants;  and 

3.  Standardized  Evaluation  Procedures. 
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Boftwar*  Installation.  PDAT-JA  software  was  installed  at  the 
U.S.  Army  Signal  Center  and  School,  Fort  Gordon  (FG) ,  Georgia, 
the  U.S.  Army  Engineer  School,  Fort  Leonard  Wood  (FLW) ,  Missouri, 
and  the  U.S.  Army  Ordnance  Center  and  School,  Aberdeen  Proving 
Ground  (APG) ,  Maryland,  for  the  purpose  of  software  evaluation. 
This  provided  a  sufficient  number  of  test  users  for  the 
assessment  of  the  capability  and  utility  of  the  prototype 
software.  FLW  and  APG  participated  as  active  evaluation 
participants  while  FG  provided  input  through  a  peer  review 
process . 

Belection  of  participants.  Selection  criteria  were  developed  to 
describe  the  expected  users  of  the  fielded  version  of  PDAT-JA. 
Evaluation  participants  were  selected  against  these  criteria. 

The  biographical  data  were  recorded  for  all  participants. 

Experience.  Evaluation  subjects  were  required  to  have  at 
least  one  year  of  experience  as  an  MOS  analyst  in  a  personnel 
proponent  agency  and  have  completed  a  minimum  of  one  MOS 
restructuring  action.  The  analysts  were  also  required  to 
demonstrate  a  good  understanding  of  the  MOS  restructuring  process 
and  have  a  good  working  knowledge  of  the  acronyms  used  in  the 
personnel  proponent  community. 

MOS  or  civilian  iob  series.  To  be  selected  as  a 
participant,  military  analysts  were  required  to  be  in  the  ranks 
of  Staff  Sergeant  to  Sergeants  Major  and  civilian  analysts  were 
to  be  in  paygrades  GS-07  to  GS-12.  Military  analysts  could  hold 
any  MOS  within  the  CMF  they  were  responsible  for.  However,  the 
only  civilian  job  series  that  were  utilized  in  the  evaluation 
were  job  series  241,  Force  Integration  Analyst;  243,  Force 
Integrator;  and  301,  Military  Occupational  Analyst. 

Profile  of  Evaluation  Participants.  Two  evaluation  participants 
consisting  of  enlisted  personnel  were  selected  from  members  of 
the  personnel  proponent  offices  at  both  the  Engineer  and  Ordnance 
Schools.  All  personnel  met  the  criteria  for  selection  and  the 
group  consisted  of  wide  ranging  military  backgrounds.  The  MOSs 
represented  by  the  evaluators  included  MOSs  51T,  63H,  63Z,  and 
81Z.  A  profile  of  this  group  is  shown  in  Table  1.  All 
participants  were  briefed  on  the  purpose  of  the  software 
evaluation  and  their  role  as  participants. 

Standardized  Evaluation.  The  evaluation  consisted  of  two  stages. 
The  first  stage  was  an  orientation  briefing;  the  second  was  the 
user  evaluation  of  the  software.  Both  are  described  below. 

Orientation  briefing.  All  personnel  selected  for 
participation  in  the  evaluation  were  provided  an  orientation 
briefing.  This  was  accomplished  at  the  time  the  software  was 
installed  at  the  selected  sites.  The  orientation  briefing  was 
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Tatolm  1 


Profil*  Data  for  Evaluation  Participants 


Number  of  Evaluators:  4 

Average  Grade:  Sergeant  First  Class 

Average  Time  as  Analyst:  32  Months 

Average  Number  of  Actions  Completed:  5 

MOSs  Represented  by  Analysts: 

5 IT  Technical  Engineering  Supervisor 

63H  Track  Vehicle  Repairer 

63Z  Mechanical  Maintenance  Supervisor 

81Z  Topographic  Engineering  Supervisor 


10 


comprised  of  (1)  an  overview  of  the  PDAT-JA  software's  design  and 
system  specifications,  (2)  an  overview  of  the  evaluation,  (3)  a 
discussion  of  the  roles  and  responsibilities  of  the  evaluation 
participants,  and  (4)  an  overview  of  the  evaluation  methodology. 

User  evaluation.  Following  the  evaluation  briefing, 
selected  MOS  analysts  used  the  PDAT-JA  software  for  three  to  four 
weeks.  During  this  time,  the  analysts  were  required  to  perform  a 
position  data  analysis  scenario  designed  to  exercise  all 
procedures  identified  in  the  PDAT-JA  User's  Manual.  The  scenario 
was  broken  down  into  seven  tasks  that  were  designed  to 
demonstrate  the  capability  of  the  end  user  to  perform  all 
functions  specified  in  the  system  software  criteria.  The 
scenario  was  arranged  in  the  logical  sequence  of  position  data 
analysis.  A  copy  of  the  scenario  is  at  Appendix  A. 

Upon  completion  of  the  scenario,  the  analysts  provided 
subjective  assessments  of  the  software's  contribution  in  terms  of 
(1)  reducing  data  manipulation  requirements,  (2)  increasing  data 
accuracy  and  reliability,  and  (3)  increasing  time  spent  on 
analysis  rather  than  data  manipulation.  Additionally,  the 
participants  were  asked  to  evaluate  the  software  in  terms  of 
general  system  design,  overall  acceptability,  data  display,  data 
entry  procedures,  and  user-control  compatibility.  A 
questionnaire  was  provided  to  the  analysts  to  support  their 
evaluation  of  the  software.  See  Appendix  B  for  a  copy  of  the 
questionnaire  used  to  obtain  these  data  concerning  PDAT-JA 
assisted  position  data  analysis. 

To  establish  a  baseline  for  measurement  of  performance 
improvements,  data  were  gathered  on  the  accomplishment  of 
position  data  analysis  in  the  current  manual  mode.  A 
questionnaire  was  used  to  solicit  input  in  terms  of  relative  time 
spent  on  each  step  of  analysis,  degree  of  difficulty,  average 
time  spent,  accuracy  and  reliability  of  analyst  produced  data,  as 
well  as  other  aspects  of  analysis  performance.  A  copy  of  the 
questionnaire  used  to  evaluate  manual  position  data  analysis  is 
provided  in  Appendix  C. 

The  analysts  were  not  required  to  fill  out  the 
questionnaires.  The  questionnaires  were  provided  to  the  analysts 
to  furnish  them  with  a  basis  from  which  to  perform  their 
assessments.  Once  the  software  was  installed  and  operated  at  the 
evaluation  sites  for  three  to  four  weeks,  interviews  of  the 
evaluation  subjects  (selected  MOS  analysts)  were  conducted  and 
the  questionnaires  completed  by  the  author. 
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PDAT-JA  Evaluation  Rasults 


The  results  of  the  PDAT-JA  evaluation  are  presented  in  this 
section.  Iriprovements  in  performance  resulting  from  introduction 
of  PDAT-JA  are  presented  for  each  analytical  step  of  position 
data  analysis.  Additionally,  the  results  of  PDAT-JA  assisted 
position  data  analysis  are  discussed  in  terms  of  the  software's 
overall  contribution  to  improving  analytical  capabilities. 

Performance  Improvements 

Baseline  evaluation  data  were  gathered  to  develop  a 
depiction  of  position  data  analysis  as  performed  in  the  manual 
mode.  Since  a  large  majority  of  personnel  proponents  are 
currently  performing  position  data  analysis  manually,  these  data 
are  representative  of  the  level  of  effort  being  expended  on  this 
type  cf  analysis  at  most  branch  proponent  schools.  These  data 
provide  the  baseline  from  which  PDAT-JA 's  contribution  to 
improving  mission  accomplishment  is  evaluated. 

Baseline  data  were  gathered  from  four  evaluation  subjects; 
two  at  APG  and  two  at  FLW.  F6  did  not  provide  baseline 
information  as  position  data  analysis  is  semi-automated  through 
the  use  of  dBase  III+  at  this  agency.  Since  the  universe  from 
which  to  gather  these  data  is  small  and  because  a  wide  variance 
in  the  responses  to  some  of  the  questions  existed,  the  data 
gathered  through  this  evaluation  are  treated  as  anecdotal  rather 
than  statistically  representative.  Accordingly,  the  baseline 
data  are  used  only  to  make  comparisons  between  the  individual 
analyst's  performance  of  position  data  analysis  in  a  manual  mode 
and  analysis  supported  by  PDAT-JA.  Notwithstanding  this 
limitation,  based  on  the  author's  personnel  proponent  experience 
over  nine  years,  the  findings  are  generally  representative  of 
personnel  proponent  experience  throughout  the  Army. 

Position  data  analysis  consists  of  several  analytical  steps. 
Each  step  has  a  distinct  purpose  and  requires  specific  data  to  be 
gathered.  Therefore,  the  evaluation  of  position  data  analysis  as 
currently  performed  is  broken  down  into  the  following  steps  or 
categories: 

1.  Application  of  the  Army's  Average  Grade  Distribution 
Matrix  (AGM) ; 

2.  Additional  skill  identifier  (ASI)  data  collection  and 
analysis; 

3.  Specialty  qualification  identifier  (SQI)  data  collection 
and  analysis; 

4.  Major  Army  command  (MACOM)  data  collection  and  analysis; 
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5.  Modified  table  of  organization  and  equipment  (MTOE) 
versus  table  of  distribution  and  allowances  (TDA)  data 
collection  and  analysis; 

6.  Continental  United  States  (CONUS)  versus  outside  the 
continental  United  States  (OCONUS)  data  collection  and 
analysis;  and 

7.  Modifications  to  TAADS  documents. 

The  baseline  or  manual  mode  data  are  compared  to  the  data 
gathered  on  PDAT-JA  assisted  position  data  analysis.  The  data 
are  compared  in  terms  of  the  relative  time  spent  gathering  data, 
the  accuracy  and  reliability  of  data,  and  the  frequency  of  data 
collection  for  each  category.  Ratings  by  the  analysts  on  the 
accuracy  and  reliability  of  data  were  based  on  the  following 
scale: 

1  -  Very  Low  Accuracy  and  Reliability 

2  -  Low  Accuracy  and  Reliability 

3  -  Less  Than  Average  Accuracy  and  Reliability 

4  -  Average  Accuracy  and  Reliability 

5  -  More  Than  Average  Accuracy  and  Reliability 

6  -  High  Accuracy  and  Reliability 

7  -  Extremely  High  Accuracy  and  Reliability 

Figure  1  summarizes  the  baseline  data  collected  on  position 
data  analysis  as  performed  in  the  manua)  mode.  As  depicted  in 
the  chart,  approximately  59  percent  of  the  analyst's  time  is 
spent  modifying  TAADS  by  making  pencil  changes  to  a  hard  copy 
TAADS  report  document.  Additionally,  almost  18  percent  of  the 
analyst's  time  is  spent  gathering  MOS  grade  structure  data  in 
order  to  determine  if  the  grade  distribution  of  the  MOS  falls 
within  the  norms  of  the  Army's  Average  Grade  Distribution  Matrix 
after  application  of  a  new  or  revised  SGA.  The  remaining  23 
percent  of  the  analyst's  time  is  spread  between  gathering  and 
evaluating  ASI  and  SQI  data,  MACOM  data,  MTOE  versus  TDA  data, 
and  CONUS  versus  OCONUS  data. 

Figure  2  depicts  the  total  time  each  evaluation  subject 
spends  performing  position  data  analysis  manually.  As 
represented  by  the  chart,  three  of  the  four  subjects  estimated 
the  position  data  analysis  process  required  between  350  to  400 
hours  to  complete  an  MOS  merger  or  restructuring  action 
consisting  of  approximately  5000  authorized  positions.  However, 
one  subject  felt  that  the  total  data  gathering  and  analysis 
process  required  over  1100  hours  to  complete.  What  is 
interesting  about  this  finding  is  that  not  only  was  this 
subject's  appraisal  of  the  time  required  for  performing  various 
analytical  steps  higher  than  the  others,  the  subject  also 
performed  each  step  more  often.  When  asked  about  this,  the 
subject  stated  that  performing  position  data  analysis  in  the 
manual  mode  was  very  time  consuming  and  mistake  prone. 
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Figure  2.  Estimated  time  spent  on  position  data  analysis  by  each 
evaluation  subject  (manual  mode) . 
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This  subject's  position  was  that  all  mistakes  rec[ulred 
Identification  and  correction  before  moving  to  the  next  step  of 
analysis.  Therefore,  an  analysis  step  can  and  usually  does 
require  considerable  time  and  multiple  iterations  to  Insure 
completeness  and  correctness.  He  stated  that  many  MOS  analysts 
subscribed  to  the  notion  that  close  is  good  enough,  and 
therefore,  did  not  adequately  perform  all  the  steps  required  in 
position  data  analysis. 

Application  of  the  AGM.  All  evaluation  participants  performed 
this  step  of  analysis  both  on  original  TAADS  data  and  on  data 
changed  by  the  analyst  after  application  of  a  new  or  revised  SGA. 
As  depicted  in  the  Pre  PDAT-JA  portion  of  Table  2,  the  process  of 
applying  the  AGM  to  an  MOS  structure  was  not  in  itself  time 
consuming.  The  time  required  to  apply  the  AGM  ranged  from  .16 
hours  (10  minutes)  to  .50  hours  (30  minutes).  However,  gathering 
data,  especially  after  application  of  a  new  or  revised  SGA 
required  from  15  to  80  hours  to  perform.  When  these  time 
requiremf'  .s  are  multiplied  by  the  number  of  times  each  analyst 
must  re\  se,  apply,  and  perform  the  counts  on  MOS  data  before  an 
acceptable  SGA  can  be  developed,  the  requirement  becomes 
significant.  The  total  time  required  for  the  AGM  process  ranged 
from  30  to  241  hours.  The  accuracy  and  reliability  of  data 
gathered  manually  for  this  analysis  ranged  between  3  (less  than 
average)  and  5  (more  than  average)  . 

All  subjects  also  performed  this  analysis  step  using  the 
PDAT-JA  software.  As  depicted  in  the  Post  PDAT-JA  section  of 
Table  2,  the  time  required  to  apply  the  average  grade  matrix  was 
reduced  to  .05  hours  (3  minutes)  or  less.  The  time  required  to 
make  counts  and  gather  data  after  application  of  a  new  or  revised 
SGA  was  also  reduced  from  15-80  hours  to  3-5  hours.  The  total 
required  for  the  AGM  process  was  significantly  decreased  from  a 
range  of  30-241  hours  in  the  manual  mode  to  6-15  hours  using 
PDAT-JA.  On  average,  the  percent  of  time  saved  by  automating 
this  step  of  position  data  analysis  is  87  percent.  The  accuracy 
and  reliability  of  data  gathered  and  analyzed  through  the 
software  was  rated  as  7  (extremely  high). 

A8I  data  collection  and  analysis.  Three  out  of  the  four 
evaluation  participants  said  they  gathered  and  analyzed  ASI  data. 
The  remaining  participant  did  not  perform  this  function  because 
only  two  low  density  ASIs  were  associated  with  the  MOSs  for  which 
he  had  responsibility.  The  analyst  stated  that  no  work  at  all  is 
done  on  either  ASI  unless  a  specific  issue  related  directly  to 
the  ASI  comes  up.  The  Pre  PDAT-JA  chart  on  Table  3  shows  ASI 
data  are  collected  at  least  twice  during  position  data  analysis 
and  the  time  spent  on  this  process  ranged  from  10  to  22  hours. 

The  total  time  required  to  perform  this  analysis  step  fluctuated 
from  20  to  44  hours  for  each  analyst.  The  accuracy  and 
reliability  of  these  data  were  rated  as  4  (average)  . 
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ASI  Datm  Collaotion  and  Analysis 
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All  analysts  performed  ASI  data  analysis  using  PDAT-JA.  As 
depicted  in  the  Post  PDAT-JA  chart,  the  time  spent  on  this 
portion  of  analysis  ranged  from  .03  to  .50  hours  (2  to  30 
minutes)  .  Based  on  the  time  spent  by  Subject  2,  this  is  a 
reduction  in  time  of  up  to  21^  hours  from  the  manual  analysis 
mode.  On  average,  time  savings  for  ASI  data  analysis  averaged  99 
percent.  Data  accuracy  and  reliability  increased  from  4 
(average)  to  7  (extremely  high) . 

SOI  data  colleotlon  and  analysis.  Again,  three  of  four 
evaluation  participants  indicated  that  they  gathered  and  analyzed 
SQI  data.  One  participant  did  not  gather  these  data  because  he 
felt  the  data  were  unimportant  to  the  position  data  analysis 
process  since  thu  MOSs  for  which  he  was  responsible  had  only  two 
SQIs  and  the  densities  of  SQI  positions  were  very  low.  The  Pre 
PDAT-JA  chart  on  Table  4  Indicates  that  SQI  data  are  collected 
twice  during  this  analysis  process.  The  total  time  required  to 
perform  this  step  ranged  from  6  to  10  hours.  The  rated  accuracy 
and  reliability  of  SQI  data  ranged  from  3  (less  than  average)  to 
6  (high) . 

Three  of  four  subjects  performed  SQI  data  analysis  using 
PDAT-JA.  The  Post  PDAT-JA  chart  shows  the  time  required  to 
perform  this  step  of  analysis  was  reduced  to  between  .06  hours  (4 
minutes)  and  two  hours.  Overall  time  savings  using  the  software 
for  SQI  analysis  averaged  89  percent.  Data  accuracy  and 
reliability  increased  to  7  (extremely  high) 

HACOM  data  collection  and  analysis.  Only  one  subject  indicated 
that  he  collected  and  analyzed  MACOM  data.  The  remaining 
subjects  stated  that  because  of  time  constraints  this  MACOM  data 
analysis  is  not  completed  unless  a  MACOM  requested  that  a 
specific  issue  be  addressed.  As  depicted  in  the  Pre  PDAT-JA 
chart  on  Table  5,  one  analyst  reports  that  the  total  time 
required  for  this  step  is  20  hours.  Accuracy  and  reliability  of 
data  are  rated  at  6  (high) . 

Two  subjects  used  PDAT-JA  to  analyze  MACOM  data  while  two 
did  not.  When  questioned,  the  subjects  that  did  not  perform 
MACOM  data  analysis  stated  that  there  were  no  MACOM  issues  to  be 
dealt  with  during  their  studies  and,  therefore,  the  MACOM 
analysis  function  was  not  used.  Post  PDAT-JA  chart  in  the  table 
indicates  a  significant  reduction  in  the  time  spent  on  performing 
MACOM  data  analysis.  However,  with  only  three  data  points  from 
which  to  evaluate  this  analysis  step,  no  practical  observations 
could  be  made. 

MTOE  versus  TDA  data  collection  and  analysis.  All  evaluation 
participants  performed  MTOE  versus  TDA  data  collection  and 
analysis.  The  first  chart  on  Table  6  shows  each  analyst  spends 
from  15  to  60  hours  performing  this  step  and  that  the  step  is 
completed  between  1  and  2  times  during  the  analysis  process.  The 
total  time  required  for  this  step  ranged  from  20  to  120  hours. 
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SUBJECT 

1 

2 

3 

4 

Perform  Analysis?  (Y/N) 

Y 

Y 

Y 

N 

Number  SQIs 

6 

4 

3 

0 

Time  (Hrs) 

.50 

1 

.03 

0 

Number  of  Times 

1 

2 

2 

TIME  SPENT 

.50 

2 

.06 

0 

Rate  Data  Accuracy  (1-7) 

7 

7 

7 
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Tabl«  5 


MACON  Data  Collaotion  and  Analysis 


PRE 

PDAT-JA 

MACOM  DATA 

SUBJECT 

SUBJECT 

SUBJECT 

SUBJECT 

1 

2 

3 

4 

Perform  Analysis?  (Y/N) 

N 

N 

Y 

N 

Time  (Mrs) 

10 

Number  of  Times 

2 

TIME  SPENT 

20 

Ra  e  Data  Accuracy  (1  -  7) 

6 

POST 

PDAT-JA 

MACOM  DATA 

SUBJECT 

SUBJECT 

SUBJECT 

SUBJECT 

1 

2 

3 

4 

Perform  Analysis?  (Y/N) 

N 

N 

Y 

Y 

Time  (Hrs) 

.03 

.05 

Number  of  Times 

1 

2 

TIME  SPENT 

.03 

.10 

Rate  Data  Accuracy  (1  -  7) 

7 

7 
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Tabltt  6 


NTOE  Versus  TDA  Data  Collaotion  and  Analysis 


PRE 

PDAT-JA 

MTOE  VS  TDA  DATA 

SUBJECT 

SUBJECT 

SUBJECT 

SUBJECT 

1 

2 

3 

4 

Perform  Analysis?  (Y/N) 

Y 

Y 

Y 

Y 

Time  (Hrs) 

60 

20 

15 

20 

Number  of  Times 

2 

2 

2 

1 

TIME  SPENT 

120 

40 

30 

20 

Rate  Data  Accuracy  (1-7) 

3 

4 

6 

5 

POST 

PDAT-JA 

MTOE  VS  TDA  DATA 

SUBJECT 

SUBJECT 

SUBJECT 

SUBJECT 

1 

2 

3 

4 

Perform  Analysis?  (Y/N) 

Y 

Y 

Y 

Y 

Time  (Hrs) 

1 

1 

1 

.50 

Number  of  Times 

1 

2 

1 

2 

TIME  SPENT 

1 

2 

1 

1 

Rate  Data  Accuracy  (1-7) 

7 

7 

7 

7 
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The  rated  data  accuracy  varied  from  3  (less  than  average)  to  6 
(high) . 

All  subjects  also  performed  this  analysis  step  using 
PDAT-JA.  The  Post  PDAT-JA  chart  shows  substantial  tine  savings 
were  gained  using  the  software  for  this  analysis  function.  The 
average  tine  savings  for  MTOE  versus  TDA  analysis  was  98  percent. 
Data  accuracy  was  increased  to  7  (extremely  high) . 

CONUB  versus  OCONUS  data  collection  and  analysis.  As  depicted  on 
the  first  chart  of  Table  7,  only  two  of  the  subjects  performed 
CONUS  versus  OCONUS  data  collection  and  analysis  in  the  manual 
mode.  The  participants  from  APG  stated  that  they  rarely 
performed  this  type  of  analysis  as  few  if  any  of  the  MOSs  in 
their  CNF  were  considered  space  imbalanced.  The  remaining 
participants  perform  this  analysis  for  all  MOSs  in  their  CMF  to 
ensure  that  no  space  imbalanced  condition  exists  within  the  MOS 
they  are  working  on.  Both  analysts  felt  that  this  analysis  step 
recpilred  20  hours  to  complete  and  that  the  process  is  usually 
performed  at  least  twice  during  position  data  analysis.  The 
total  time  required  for  CONUS  versus  OCONUS  data  collection  is  40 
hours.  Data  accuracy  and  reliability  ranged  from  4  (average)  to 
6  (high) . 

The  subjects  from  FLW  performed  CONUS  versus  OCONUS  data 
analysis  using  the  PDAT-JA  software.  The  second  chart  on  the 
table  reveals  a  major  decrease  in  the  time  required  for  this 
analysis  process.  On  average,  the  time  savings  between  the 
manual  method  and  that  assisted  through  PDAT-JA  was  99  percent. 
Data  accuracy  and  reliability  also  increased  to  7  (extremely 
high) . 

Modifications  to  TAAD8  documentation.  All  evaluation 
participants  made  manual  modifications  to  hard  copy  TAADS 
documentation.  As  shown  in  the  Pre  PDAT-JA  chart  on  Table  8,  the 
time  spent  for  each  iteration  of  this  function  ranged  from  60  to 
360  hours.  The  number  of  times  this  function  is  performed  varies 
from  2  to  4.  The  total  time  spent  making  modifications  to  TAADS 
fluctuated  between  a  low  of  160  hours  to  a  high  of  720  hours. 

The  accuracy  and  reliability  of  the  TAADS  data  after  manual 
modification  was  rated  between  a  low  of  2  (low)  and  a  high  of  4 
(average) . 

All  four  evaluation  subjects  used  PDAT-JA  to  make 
modifications  to  TAADS  in  the  automated  environment.  The  second 
chart  on  the  table  demonstrates  a  profound  time  savings  in 
modifying  TAADS  documentation.  The  average  "Time  Spent" 
performing  manual  modifications  to  TAADS  by  all  subjects  was  345 
hours,  while  the  average  of  "Time  Spent"  performing  modifications 
to  TAADS  using  the  PDAT-JA  software  was  80  hours.  Performing 
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Tabla  7 


CONUS  Varsus  OCONUS  Data  Collaotion  and  Analysis 


PRE 

PDAT-JA 

CONUS  VS  OCONUS  DATA 

SUBJECT 

SUBJECT 

SUBJECT 

SUBJECT 

1 

2 

3 

4 

Perform  Analysis?  (Y/N) 

N 

N 

Y 

Y 

Time  (Hrs) 

20 

20 

Number  of  Times 

2 

2 

TIME  SPENT 

40 

40 

Rate  Data  Accuracy  (1-7) 

6 

4 

POST 

PDAT-JA 

CONUS  VS  OCONUS  DATA 

SUBJECT 

SUBJECT 

SUBJECT 

SUBJECT 

1 

2 

3 

4 

Perform  Analysis?  (Y/N) 

N 

N 

Y 

Y 

Time  (Hrs) 

.50 

.04 

Number  of  Times 

1 

2 

TIME  SPENT 

.50 

.08 

Rate  Data  Accuracy  (1-7) 

7 

7 
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T«bl«  8 


Modi float ions  to  TAADS  Dooumantation 


MODIFICATIONS  TO  TAADS 

SUBJECT 

PRE 

SUBJECT 

PDAT-JA 

SUBJECT 

SUBJECT 

1 

2 

3 

4 

Perfora  Analysis?  (Y/N) 

Y 

Y 

Y 

Y 

Time  (Hrs) 

360 

80 

80 

60 

Number  of  Times 

2 

2 

4 

3 

TIME  SPENT 

720 

160 

320 

180 

Rate  Data  Accuracy  (1-7) 

2 

4 

3 

4 

POST 

PDAT-JA 

MODIFICATIONS  TO  TAADS 

SUBJECT 

SUBJECT 

SUBJECT 

SUBJECT 

1 

2 

3 

4 

Perform  Analysis?  (Y/N) 

Y 

Y 

Y 

Y 

Time  (Hrs) 

24 

24 

30 

40 

Number  of  Times 

2 

3 

4 

2 

TIME  SPENT 

48 

72 

120 

80 

Rate  Data  Accuracy  (1-7) 

7 

7 

7 

7 
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modifications  to  TAADS  in  the  automated  mode  provided  an  average 
time  savings  of  265  hours  per  subject  or  77  percent  of  the  time 
recjuired  to  perform  this  procedure  in  a  manual  mode. 

Aeeuracv  and  reliability  of  data.  The  accuracy  and  reliability 
of  data  was  also  improved  through  the  use  of  PDAT-JA.  On 
average^  accuracy  and  reliability  of  data  gathered  in  the  manual 
mode  was  4  (average)  .  On  the  other  hand,  PDAT-JA  assisted 
analysis  increased  accuracy  and  reliability  of  data  to  7 
(extremely  high) .  This  finding  is  notable  because  the  increase 
in  accuracy  can  also  be  translated  into  time  saved.  The  increase 
to  extremely  high  accuracy  and  reliability  means  that  time  is  not 
lost  going  bac]c  over  work  to  find  mistakes;  and  in  many  cases, 
repeating  entire  analytical  steps  because  the  errors  cannot  be 
found  is  almost  completely  eliminated.  During  the  course  of 
position  data  analysis,  elimination  of  mistakes  can  mean  savings 
of  hundreds  of  hours,  especially  for  the  novice  analyst. 

summary  Of  Improvements 

Based  on  the  findings  described  above,  there  is  no  question 
that  PDAT-JA  enhances  the  analyst's  capability  to  perform 
position  data  analysis.  However,  since  the  sample  size  is  small, 
one  must  be  very  careful  on  making  claims  of  Increased  efficiency 
without  statistical  validation.  The  only  valid  comparisons  that 
can  be  made  are  those  of  comparing  the  time  required  by  each 
subject  to  perform  position  data  analysis  in  the  manual  mode  to 
the  time  spent  performing  analysis  assisted  by  PDAT-JA. 

To  make  this  comparison,  the  total  number  of  hours  required 
by  each  subject  to  perform  position  data  analysis  in  the  manual 
mode  was  treated  as  100  percent  of  "time  required"  regardless  of 
the  actual  time  required  for  analysis.  The  "time  spent"  by  each 
subject  performing  PDAT-JA  assisted  analysis  for  the  same  number 
of  MOS  positions  was  then  compared  to  the  "time  required". 

Figure  3  depicts  the  comparison.  The  solid  black  bars  on 
the  chart  represent  the  "time  required"  for  each  subject  to 
perform  position  data  analysis  manually.  The  shaded  bars 
represent  the  "time  spent"  by  the  subjects  performing  analysis 
using  PDAT-JA.  In  every  case,  the  time  saved  through  the 
utilization  of  PDAT-JA  is  substantial. 

Figure  4  provides  a  representation  of  the  percent  of  total 
time  saved  by  each  evaluation  subject  while  performing  position 
data  analysis  using  PDAT-JA  versus  the  manual  mode.  As  depicted 
in  the  figure,  overall  time  savings  ranged  between  72  and  95 
percent.  The  savings  in  time  and  effort  establish  the 
credibility  of  PDAT-JA  as  a  viable  enhancement  to  the  process  of 
position  data  analysis  for  those  functions  described  in  the 
PDAT-JA  software  standards. 
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Figure  3.  Comparison  of  manual  position  data  analysis  (time 
required)  versus  analysis  assisted  through  PDAT-JA 
(time  spent) . 


28 
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Figure  4.  Percent  of  total  time  saved  through  using  PDAT-JA. 
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Rcoonmandcd  PDAT-JA  Software  Nodifioations 


The  purpose  of  this  section  is  to  review  the  recommended 
modifications  to  the  PDAT-JA  software  provided  by  the  evaluation 
participants.  The  recommended  modifications  are  evaluated 
against  the  criteria  and  constraints  used  for  developing  PDAT-JA. 
Based  upon  this  evaluation,  final  software  modification 
recommendations  are  identified. 

improvements  Racommendad  bv  Test  Users 

The  following  is  a  discussion  of  the  improvements  to  the 
PDAT-JA  software  recommended  by  the  evaluation  subjects  from  FLW, 
APG,  and  FG.  Table  9  provides  a  list  of  improvements  along  with 
the  estimated  level  of  effort  to  perform  each. 

Recommendation  l:  Expand  data  import  capability.  The  software 
should  be  capable  of  importing  data  from  a  magnetic  tape  and 
storing  the  data  in  a  central  file  server  that  can  be  accessed 
over  a  mini-computer  based  local  area  network.  The  Signal 
proponent  stated  that  this  type  of  hardware  capability  should  be 
considered  the  standard  mode  of  operation  across  proponent 
schools.  The  rationale  for  this  recommendation  is  based  on  the 
experience  of  the  Signal  School  that  the  length  of  time  required 
to  download  Headquarters  Department  of  the  Army  (HQDA)  data  is 
considerable.  The  Signal  proponent  ordered  TAADS  data  from  HQDA 
on  MOSs  31F  and  36M.  Taken  together,  these  MOSs  consist  of 
approximately  1500  authorized  positions.  The  reviewer  indicated 
that  a  period  of  two  weeks  was  required  to  receive  the  data  and 
that  21  floppy  diskettes  were  needed  to  store  the  TAADS  data. 

Once  the  data  were  received,  the  reviewer  stated  that  almost  two 
hours  were  required  to  download  and  process  the  data  using  PDAT- 
JA. 


Currently,  the  Signal  Proponent  has  hardware  capability  to 
download  TAADS  from  magnetic  tape  to  a  minicomputer.  The 
proponent  also  uses  dBase  III+  as  their  primary  data  base 
management  software.  The  reviewer  maintains  that  the  data  for 
MOSs  3  IF  and  36M  could  be  downloaded  in  approximately  40  minutes 
using  dBase  III+.  Therefore,  the  use  of  PDAT-JA,  as  currently 
configured,  is  comparatively  time  consuming  in  terms  of  ordering 
data  from  HQDA  and  downloading  the  data  into  a  personal  computer. 

Finding.  One  of  the  design  objectives  for  PDAT-JA  was  that 
the  software  operate  on  Army  standard  PC  equipment  and  be  both 
exportable  and  nonproprietary.  The  current  state  of  the  Signal 
School's  computer  hardware  capability  far  exceeds  that  found  at 
other  Army  schools.  During  the  development  of  PDAT-JA,  several 
schools  were  surveyed  to  determine  the  current  state  of  hardware 
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T«bl«  9 


Raquastttd  Modifications  to  PDAT-JA 


Rsquastod  Modifications  Estimated  Hours 

1.  Revise  Data  Import .  20 

2.  Provide  Global  MOS  Duty  Title  Replacement .  40 

3.  Increase  the  Three  MOS  Analysis  Limitation .  240 

4a.  Reduce  the  Number  of  Editing  Steps  Required .  40 

4b.  Remove  Editing  Window .  180 

4c.  Change  Default  Settings .  20 

4d.  Remove  the  "Authorizations  Must  be  Less  Than 

Requirements"  Prohibition .  * 

5.  Provide  PMAD  Data  Import  and  Analysis  Capability.  200 

6.  Provide  UIC  Level  Requirements  and 

Authorizations  Delete  Function .  20 

7.  Provide  Global  MOS  Edit  Function .  120 


Total  Estimated  Hours  880 


*  Hours  included  with  4b. 
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capability.  Some  of  the  schools  contacted  had  no  computer 
capability,  while  most  others  had  the  Army  standard  system.  The 
only  exceptions  were  the  Signal  School  and  the  U.S.  Army  Academy 
of  Health  Sciences  which  both  have  exceptional  capability. 

HQDA  is  sometimes  slow  responding  to  requests  for  TAADS  data 
regardless  of  what  transfer  media  are  requested.  However,  one 
must  recognize  that  requests  which  require  data  to  be  downloaded 
on  5-1/4”  diskette  as  an  ASCII  file  are  not  currently  the 
standard  way  data  are  downloaded  by  HQDA.  Therefore,  at  least 
Initially,  requests  may  take  a  little  longer  to  execute. 

The  requirement  for  21  diskettes  to  download  NOSs  3  IF  and 
36M  cannot  be  explained.  These  MOSs  are  very  small  and  should 
only  require  two  or  three  diskettes  for  storage.  However,  one 
might  speculate  that  the  download  contained  more  TAADS  data  than 
requested  (e.g.,  the  entire  CMF  rather  than  the  requested  MOSs). 

During  the  data  import  process,  PDAT-JA  is  designed  to 
condition  data  as  it  is  being  read  into  the  data  base.  The 
conditioning  process  ensures  that  incomplete  data  or  data  that  do 
not  contain  proper  MOS  coding  are  not  read  into  the  PDAT-JA  data 
base.  To  compare  PDAT-JA 's  time  to  perform  this  task  with  that 
of  dBase  Ill-f,  which  only  reads  the  data  into  a  data  base  without 
the  same  data  conditioning,  analysis,  or  report  file  generation, 
is  not  a  valid  comparison. 

A  less  significant  data  import  modification  emerged  from  the 
evaluation.  This  entails  providing  the  capability  to  function 
with  additional  disk  drive  designators  besides  the  conventional 
A:\  and  C:\  disk  drives  within  PDAT-JA's  import  function.  This 
modification  can  easily  be  accomplished  and  would  have  general 
applicability  throughout  the  user  community. 

Implementation  evaluation.  The  Signal  proponent's 
recommendation  would  lead  to  a  customization  of  the  PDAT-JA 
software  that  exceeds  the  design  guidelines  for  the  software. 

In  order  for  PDAT-JA  to  download  data  from  a  non-formatted 
9-track  tape,  a  transport  and  interface  card  are  necessary  and 
specialized  software  is  required.  Additionally,  a  FOXPro 
interface  software  program  which  pre-processes  the  data  from  the 
tape  would  also  be  required.  However,  as  the  state  of  capability 
evolves  and  bigger  and  better  computers  are  installed  throughout 
the  Army,  this  recommendation  may  need  to  be  reassessed. 

On  the  other  hand,  adding  additional  drive  designators  is  a 
modification  than  can  easily  be  accomplished  with  minimal  effort. 
The  change  would  essentially  involve  redesign  of  the  effected 
data  import  screens  and  involve  an  estimated  20  hours. 
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Reviewers  at  F6  and  FLW  recommended  that  PDAT-JA  be  given  the 
capability  to  globally  replace  MOS  information  in  the  TAADS 
worksheet  rather  than  having  to  make  individual  changes  to  each 
line  of  data.  The  rationale  for  this  recommendation  is  that  in 
many  cases  an  MOS  action  will  involve  a  title  change,  often 
consisting  of  only  one  phrase  or  word.  Since  PDAT-JA  requires 
each  line  of  data  to  be  Individually  edited  and  updated,  the 
reviewers  felt  the  process  was  too  time-consuming. 

Finding.  A  global  replacement  capability  is  needed  to 
reduce  the  time  spent  performing  repetitive  changes  to  TAADS  data 
while  editing  the  PDAT-JA  TAADS  worksheet.  For  example,  one 
function  could  be  to  change  all  MOS  titles  based  on  a  particular 
grade  or  duty  title. 

Implementation  evaluation.  A  global  replacement  capability 
can  be  developed  for  PDAT-JA  keyed  on  MOS,  duty  title,  and  grade. 
However,  replacement  of  all  titles  would  be  made  by  matching  the 
exact  spelling  of  the  title  specified  by  the  user.  Therefore,  if 
titles  exist  in  several  forms  in  TAADS  (words  abbreviated  or  even 
misspelled) ,  a  global  replacement  would  require  all  variations  of 
the  original  'title  to  be  entered.  For  example,  USA  and  U.S.A. 
and  UAS,  if  a  typographical  error  existed  in  the  original  data, 
would  need  to  be  entered  by  the  user  to  effect  the  replacement. 

A  menu  option  could  be  added  to  the  existing  program  to  provide 
this  feature. 


Recommendation  3:  Increase  the  three  MOS  analysis  limitation. 
Currently,  PDAT-JA  provides  analysis  and  worksheet  development 
capability  for  three  MOSs.  Reviewers  at  FG  and  FLW  stated  that 
MOS  consolidation  actions  could  easily  require  the  analysis  of 
more  than  three  MOSs  during  a  restructuring  action  and  therefore 
this  limitation  should  ue  removed. 


Finding.  The  analysis  and  TAADS  worksheet  capabilities  of 
PDAT-JA  were  intentionally  limited  to  three  MOSs  because  of 
hardware  data  storage  constraints.  On  average,  the  Army  standard 
PC  has  only  20  Mbytes  of  storage  capacity  on  a  single  hard  drive. 
PDAT-JA  is  designed  to  read  into  its  data  base  both  the  target 
MOS  authorizations  and  all  other  MOS  authorizations  that  reside 
in  the  same  TAADs  paragraph  as  the  target  MOS.  Therefore,  the 
number  of  MOS  authorizations  in  the  original  TAADS  data  must  be 
limited  to  approximately  5000  authorizations.  PDAT-JA  performs 
analysis  and  writes  files  for  use  in  report  generation;  these 
files  take  up  space  on  the  hard  disk.  Additionally,  PDAT-JA 
allows  the  user  to  develop  an  automated  worksheet  that  is  an 
exact  duplicate  of  the  original  data  downloaded  during  the  import 
process.  This  worksheet  is  used  by  the  analyst  to  make  changes 
to  TAADS  based  on  a  new  or  revised  SGA.  The  worksheet  takes  up 
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additional  space  on  the  disk  equal  to  that  of  the  original  data. 
Given  this,  the  developers  of  PDAT-JA  limited  the  number  of  MOSs 
the  software  would  accept  to  three,  and  set  the  recommended 
maximum  MOS  authorizations  at  5000  as  a  way  of  protecting  the 
user  from  potential  system  failure  associated  with  hardware 
constraints. 

Implementation  evaluation.  Given  the  hardware  limitations 
of  the  current  Army  standard  PC,  action  on  this  recommendation 
should  be  deferred  until  hardware  capabilities  and  storage 
capacities  are  upgraded.  As  a  rule-of-thumb,  the  Army  standard 
PC  system  would  need  to  be  upgraded  to  at  least  an  80386  central 
processing  unit  and  60  Mbytes  of  internal  storage  capacity  before 
considering  an  upgrade  of  the  software  to  accept  a  larger  number 
of  MOSs.  Further,  many  of  PDAT-JA 's  input  and  display  screens 
as  well  as  a  significant  amount  of  the  programming  and  reports 
are  based  on  a  three-MOS  input  and  data  analysis  limit.  To 
change  this  code,  would  require  significant  modifications 
throughout  almost  every  aspect  of  the  program. 

Recommendation  4:  Modify  PDAT-JAs  editing  mode.  The  FG  reviewer 
made  four  recommendations  for  modifying  PDAT-JA  capabilities  to 
edit  the  PDAT-JA  TAADS  Worksheet.  These  recommendations  are 
discussed  separately. 

4a.  Reduce  the  number  of  editing  steps  required.  The 
reviewer  recommended  that  the  number  of  steps  required  to  perform 
editing  be  reduced.  When  editing  a  worksheet,  the  analyst  must 
move  from  the  data  base  window  to  the  worksheet  window,  call  up 
the  editing  window,  make  changes  in  the  editing  window  to  the 
worksheet,  and  exit  the  editing  window  when  the  changes  are 
complete.  To  the  reviewer,  this  process  is  overly  cumbersome  and 
time  consuming. 

Finding.  The  number  of  steps  required  to  perform  work  in 
the  editing  mode  is  a  bit  cumbersome  and  time  consuming.  In 
part,  the  problem  is  one  of  getting  used  to  the  software.  On  the 
other  hand,  software  design  plays  a  role  in  this  issue.  The 
editing  window  is  designed  to  reduce  the  probability  of  making 
mistakes.  This  is  accomplished  through  a  series  of  checks  and 
balances  programmed  into  the  edit  feature  of  PDAT-JA  for  quality 
control  purposes.  For  example,  the  edit  program  examines  the 
analyst's  input  to  ensure  that  paygrade  and  skill  level  match. 
Additionally,  the  program  will  not  allow  the  analyst  to  exit  the 
edit  mode  unless  the  TAADS  paragraph  being  edited  is  in  balance. 
This  feature  is  executed  by  ensuring  that  the  total  of  the  MOS 
authorizations  revised  by  the  analyst  do  not  exceed  the  total 
number  of  original  MOS  authorizations. 
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without  the  preprogramned  checks  and  balances,  the  analyst 
can  make  mistakes  while  editing  and  not  have  any  indication  an 
error  was  made.  The  time  spent  performing  these  extra  steps 
while  in  the  edit  mode  can  save  hours  of  back  tracking  through 
the  worksheet  looking  for  mistakes. 

Implementation  evaluation.  PDAT-JA  can  be  reprogrammed  so 
that  the  editing  feature  is  not  so  cumbersome.  However,  to  do 
this  would  necessitate  the  removal  of  the  preprogrammed  quality 
control  features  or  the  addition  of  a  separate  editing  mode 
without  a  quality  control  feature. 

4b.  Remove  editing  window.  The  editing  window  covers  the 
TAADS  window  on  the  screen  when  in  the  editing  mode  and  thus 
TAADS  cannot  be  referred  to  when  editing.  The  FG  reviewer 
recommends  that  the  editing  window  be  removed  and  that  editing  be 
allowed  in  the  TAADS  worksheet. 

Finding.  The  editing  window  does  obstruct  the  analyst  view 
of  the  TAADS  window.  However,  to  remove  the  editing  window  would 
result  in  the  same  impact  as  outlined  in  4a  above. 

Implementation  evaluation.  The  editing  window  can  be 
removed  and  provisions  made  to  directly  edit  the  TAADS  worksheet. 
However,  this  would  also  result  in  the  loss  of  the  quality 
control  features  designed  into  PDAT-JA.  For  the  inexperienced 
analyst,  removal  of  the  editing  window  would  provide  the 
likelihood  of  making  unrecoverable  mistakes  as  no  safeguards 
would  be  in  place. 

One  solution  could  be  to  provide  a  menu  option  to  select 
editing  through  the  editing  window  or  editing  within  the 
worksheet  window.  However,  because  of  radical  differences 
between  these  two  methods,  difficulties  would  arise  when  users 
try  to  switch  from  one  method  to  the  other.  Another  option  might 
be  to  simply  reposition  the  editing  window  to  allow  a  full  view 
of  the  original  data  base  window. 

4c.  Change  default  settings.  When  exiting  from  th^  editing 
window,  PDAT-JA  returns  the  user  to  the  TAADS  window  and  not  the 
worksheet.  The  reviewer  recommends  that  since  the  worksheet 
contains  the  data  being  edited,  the  program  should  default  to  the 
worksheet  when  exiting  the  edit  mode. 

Finding.  PDAT-JA  is  designed  to  default  to  the  TAADS  window 
rather  than  the  worksheet.  Most  data  base  management  systems 
require  the  user  to  SAVE  changes  before  the  data  base  is  actually 
updated.  Likewise,  PDAT-JA  also  requires  a  SAVE  command  be 
executed  before  the  worksheet  is  actually  updated.  The  software 
executes  a  SAVE  by  taking  the  user  out  of  the  worksheet  editing 
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mode  back  to  the  TAADS  window.  The  PDAT-JA  TAAOS  data  base  and 
the  TAADS  worksheet  are  linked  in  terms  of  movement  through  the 
data  bases.  Another  reason  that  PDAT-JA  defaults  to  the  TAADS 
window  is  that  the  worksheet  is  keyed  on  the  TAADS  data  base  by 
the  use  of  data  hooks.  These  data  hooks  allow  the  user  to  select 
a  unit  identification  code  (UIC)  and  paragraph  in  the  TAADS 
window  and  the  software  automatically  advances  the  worksheet  to 
the  same  UIC  and  paragraph.  Without  this  feature  comparisons 
between  the  original  TAADS  data  and  the  worksheet  could  not  be 
made. 


Implementation  evaluation.  The  default  to  the  TAADS  window 
is  required  when  exiting  the  editing  window  in  order  to  execute  a 
SAVE  and  for  the  program  to  "refind”  its  place  in  the  TAADS 
worksheet.  However,  this  step  can  be  made  transparent  to  the 
user  by  adjusting  the  program  to  return  automatically  to  the 
TAADS  worksheet  once  the  SAVE  and  "refind"  commands  have  been 
executed . 

4d.  Remove  the  «»authorigationa  must  be  lass  than 
reauiramenta**  prohibition.  The  software  is  programmed  to 
prohibit  MOS  requirements  from  being  less  than  MOS  authorizations 
at  the  paragraph  level  of  detail.  The  F6  reviewer  recommended 
that  this  limitation  be  revised  to  allow  authorizations  and 
requirements  imbalances  at  the  paragraph  level  but  not  at  the  UIC 
level . 


Finding.  A  capability  to  allow  MOS  requirement  and 
authorization  imbalances  at  the  paragraph  level  is  needed  to 
accommodate  Signal's  unique  operational  requirements,  although 
regulatory  guidance  expressly  forbids  documented  authorizations 
to  exceed  requirements  except  under  two  very  narrow  exceptions 
(see  Army  Regulation  (AR)  310-49,  The  Army  Authorization 
Documents  System.  Chapter  3,  Para  3-4). 

Implementation  evaluation.  PDAT-JA  can  be  reprogrammed  to 
allow  requirements  to  exceed  authorizations.  Although  this 
modification  can  be  accomplished,  one  must  question  the  wisdom  of 
designing  software  based  on  an  exception  rather  than  the  rule. 

Recommendation  5. _ Provide  PMAD  data  import  and  analysis 

capability.  The  FG  reviewer  recommended  that  PDAT-JA  be 
furnished  with  the  capability  to  import  and  analyze  Personnel 
Management  Authorization  Document  (PMAD)  data.  The  rationale  for 
this  recommendation  is  that  the  use  of  PMAD  data  is  necessary  for 
planning  future  force  structure  changes.  SGA  revisions,  MOS 
mergers,  and  MOS  restructures  are  performed  based  on  what  the 
force  structure  is  projected  to  be  in  the  out-years  rather  than 
on  what  exists  now  or  in  the  immediate  future. 
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Finding.  Position  data  analysis  rt^^juires  comparison  between 
TAADS  and  PMAD  data  for  the  purposes  of  identifying  increases  or 
decreases  in  projected  MOS  authorizations.  This  capacity  would 
enhance  the  capability  of  PDAT-JA  by  providing  the  user  with  a 
facility  to  identify  MOS  authorization  variance,  documenting  the 
units  in  which  variance  occurs,  and  for  prompting  determination 
of  the  causes  for  variance. 

Implementation  evaluation.  The  capability  to  import  and 
compare  PMAD  data  with  TAADS  was  within  the  scope  of  the  original 
PDAT-JA  effort.  However,  PMAD  data  were  not  available  in  the 
format  needed  for  import  into  PDAT-JA.  In  lieu  of  the  PMAD 
capability,  other  features  were  added  to  the  software  that  were 
not  in  the  original  specifications.  If  data  are  now  available, 
the  PMAD  comparison  function  could  be  added  to  PDAT-JA  within  the 
scope  of  this  effort. 

Recommendation  6.  Provide  UIC  level  requirements  and 
authorigations  delete  function.  AP6  recommends  that  a  function 
that  would  permit  the  removal  of  MOS  requirements  and 
authorizations  at  the  UIC  level  be  added.  This  function  should 
allow  specific  UICs  to  be  entered  into  PDAT-JA  and  the 
requirements  and  authorizations  to  be  zeroed  out  automatically. 

Finding.  This  function  is  needed  to  provide  a  way  to  remove 
requirements  and  authorizations  from  units  that  are  not  projected 
to  be  in  the  force  structure  in  the  out-years.  Once  the 
requirements  and  authorizations  for  UICs  that  are  not  programmed 
are  removed,  position  data  analysis  and  SGA  development  can  be 
accomplished  based  on  the  projected  rather  than  current  force 
structure.  This  capability  is  important  as  MOS  actions  must 
address  future  as  well  as  current  requirements. 

Implementation  evaluation.  A  capability  to  delete 
requirements  and  authorizations  at  the  UIC  level  can  be 
accomplished  with  minimal  effort  as  a  menu  option  in  the  MPCTAADS 
worksheet. 

Recommendation  7.  Provide  qlobax  MOS  edit  function.  APG 
recommends  that  the  software  have  capability  to  perform  global 
changes  to  MOS  specifications.  For  example,  replace  MOS  91B, 

9 1C,  and  9 ID,  with  MOS  9 IE.  This  function  would  replace  only  the 
first  three  alpha  numeric  characters  of  the  MOS  code  and  not 
change  any  other  characters  in  the  MOS  data  set. 

Finding.  A  global  replacement  capability  is  needed  to 
reduce  the  time  spent  performing  repetitive  MOS  data  changes 
while  editing  the  MPCTAADS  worksheet. 
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1«p1— ntatlon  •valuation.  This  recommendation  could  be 
Implemented  by  creating  a  separate  menu  option  that  will  provide 
this  capability  in  the  import  data  function  of  the  software. 

This  would  allow  the  global  replacement  of  the  first  three 
characters  of  MOS  codes  prior  to  performing  analysis  on  the  data. 

Modifiostions  to  the  PDAT-JA  Users  Manual.  Depending  on  which 
software  changes  are  selected,  the  PDAT-JA  Users  Manual  will  be 
modified  accordingly.  No  modifications  to  the  manual  were 
recommended  by  the  evaluation  subjects. 

Final  software  Modification  Recommendation 

Because  the  level  of  effort  currently  available  for 
improving  PDAT-JA  is  limited,  the  best  solution  would  be  to 
implement  a  combination  of  recommendations  which  provide  the 
greatest  payoff  to  the  Army  as  a  whole  within  those  funding 
constraints.  Based  on  those  criteria,  recommendations  1,  2,  6, 
and  7  are  selected  as  ones  to  be  executed.  This  combination  of 
recommendations  can  be  accomplished  within  the  allotted  effort 
and  provide  Army-wide  benefits.  Items  2,  6,  and  7  will  provide  a 
more  global  editing  capability.  Item  1  would  allow  users  to  more 
easily  access  data  from  sources  having  different  disk  drive 
designators. 
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rutur«  Efforts 


PDAT-JA  software  Is  a  prototype  that  was  developed  to 
establish  the  feasibility  of  producing  automated  methods  to 
support  MOS  restructuring.  This  feasibility  and  the  demonstrated 
usefulness  of  the  prototype  suggest  that  the  Army  may  want  to 
consider  expanding  the  operational  capabilities  of  the  PDAT-JA. 
These  options  include  developing  position  data  analysis 
capabilities  for  (1)  active  duty  officers,  warrant  officers,  and 
civilians;  and  (2)  the  reserve  component  and  the  National  Guard. 

capability  te  inert  aad  Aaalvie  Active  Putv  Officer.  Warraat 
Officer,  and  Civilian  Data 

Currently,  PDAT-JA  can  only  import  and  perform  analysis  on 
enlisted  personnel  data.  In  terms  of  prototype  development  this 
capability  was  adequate  to  demonstrate  how  such  a  capability 
would  operate.  For  the  purpose  of  designing  and  developing  the 
prototype  software,  enlisted  data  were  chosen  because  enlisted 
MOS  restructuring  actions  are  the  preponderance  of  work 
accomplished  in  personnel  proponent  agencies. 

Officer,  warrant  officer,  and  civilian  specialties  also 
require  restructuring  in  the  same  manner  as  the  enlisted  force. 
Capability  to  support  officer,  warrant  officer,  and  civilian 
actions  is  needed  in  order  for  the  proponent  to  perform  analysis 
on  the  entire  active  Army  force  structure.  Although  the  current 
capabilities  of  PDAT-JA  represent  an  important  step  forward  in 
reducing  the  workload  of  the  proponents,  thousands  of  hours  will 
continue  to  be  wasted  if  officer,  warrant  officer,  and  civilian 
position  data  analyses  are  not  automated. 

Capability  to  Import  and  Analyze  Reserve  Component  and  National 
Guard  Data 

PDAT-JA  as  currently  designed  will  only  import  and  analyze 
data  on  the  active  component  of  the  Army.  During  the  process  of 
an  MOS  restructuring  action,  the  personnel  proponents  must  also 
perform  position  data  analysis  on  reserve  component  and  National 
Guard  data  to  identify  and  assess  the  effects  of  a  restructuring 
action.  This  analysis  is  conducted  using  methods  identical  to 
those  used  for  the  active  force.  If  anything,  the  time  required 
for  analysis  of  the  reserve  components  is  even  higher  than  that 
of  the  active  force  as  the  reserve  components  constitute  almost 
two  thirds  of  the  total  Army  force  structure.  If  projected 
across  all  personnel  proponent  agencies,  automation  of  reserve 
and  National  Guard  position  data  analysis  could  potentially  save 
thousands  of  hours  per  year. 

Further  Evaluation  of  PDAT-JA 

The  anecdotal  results  of  this  evaluation  demonstrate  that 
PDAT-JA  promises  to  be  a  very  robust  and  indispensable  asset  to 
the  personnel  proponent  community.  If  continued  use  of  this 
prototype  confirms  the  results  of  this  effort,  a  very  strong  case 
could  be  made  for  full-scale  development  and  fielding  of  PDAT-JA. 
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APPEHDZZ  A 


SCEMARIO 


You  are  to  use  PDAT-JA  to  assist  you  in  performing  position 
data  analysis  on  up  to  three  MOSs  which  contains  from  3,000  to 
5000  authorized  positions.  You  will  first  analyze  TAADS  data 
based  on  the  current  MOS(s)  force  structure.  You  will  make 
modifications  to  the  MOS(s)  force  structure  and  perform  position 
data  analysis  to  ascertain  what  changes  occurred  as  a  result  of 
your  modifications.  You  are  to  respond  to  the  questions  based 
only  on  the  steps  performed  in  position  data  analysis.  Time  or 
effort  spent  in  performance  of  SGA  development,  personnel  data 
analysis,  physical  demands  analysis,  etc.,  are  not  to  be 
considered  in  your  response. 

There  are  five  basic  tasks  required  in  this  scenario. 
However,  both  Tasks  4  and  5  must  be  performed  twice  (once  for 
TAADS  data  and  once  for  modified  TAADS  data)  to  complete  one 
position  data  analysis  cycle.  Please  use  the  PDAT-JA  Users 
Manual  to  help  you  complete  each  task. 

Task  It  Install  PDAT-JA.  PDAT-JA  will  be  installed  on  the 
personal  computer  (PC)  using  the  installation  and  start-up 
procedures  as  described  in  the  PDAT-JA  Users  Manual. 

Task  2:  import  TAADS  data  and  create  the  reference  data 
base.  Use  PDAT-JA  to  import  TAADS  MOS  data  and  create  the 
"PCTAADS  Referenae”  data  base.  It  is  recommended  that  up  to 
three  MOS(s)  with  a  maximum  of  5000  authorized  positions  be  used 
for  this  task.  MOS(s)  with  more  authorizations  could  exceed  the 
data  storage  capacity  of  the  PC  or  make  it  to  difficult  to 
perform  all  the  tasks  required  in  this  scenario. 

Task  3: _ create  an  MPCTAAD8  worksheet  and  modify  MOS  force 

structure  data.  Create  the  "NPCTAADS  Worksheet"  using  the 
instructions  in  the  Users  Manual.  Use  the  worksheet  to  modify 
data  on  a  minimum  of  3000  authorized  positions. 

Task  4:  Generate  siimroarv  reports.  Use  PDAT-JA  to  perform 
analysis  on  both  PCTAADS  and  MPCTAADS  data.  Upon  completion  of 
analysis,  generate  the  following  reports: 

■  Summary  of  MOS  Authorizations  by  Grade; 

■  Summary  of  Additional  Skill  Identifier  (ASX) 
Authorizations  by  Grade; 

■  Summary  of  Specialty  Qualification  Identifier  (SQI) 
Authorizations  by  Grade; 
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■  Summary  of  Major  Army  Command  (MACOM)  MOS 
Authorizations  by  Grade; 

■  Summary  of  MOS  Authorizations,  Modified  Table  of 
Organization  and  Equipment  (MTOE)  versus  Table  of 
Distribution  and  Allowance  (TDA)  by  Grade; 

■  Summary  of  MOS  Authorizations,  Continental  United 
States  (CONUS)  versus  Outside  of  Continental  United 
States  (OCONUS)  by  Grade; 

■  MOS  Profile  Status  Report; 

■  PCTAADS  and  MPCTAADS  Data  Base  Reports;  and 

■  PCTAADS  and  MPCTAADS  Comparative  Report. 

Task  5t _ Perform  grade  structure  analysis.  Use  PDAT-JA  to 

perform  grade  structure  analysis  on  both  PCTAADS  and  MPCTAADS 
data  and  produce  graphic  representations  of  both  the  MOS  grade 
structure  and  the  percent  delta  between  the  MOS  and  the  Army's 
Average  Grade  Distribution  Matrix. 

If  you  were  not  able  to  complete  all  tasks  in  the  time 
allotted,  please  inform  the  interviewer  of  how  much  work  was 
completed  and  the  reasons  for  not  completing  the  scenario.  If 
the  scenario  was  not  completed,  please  respond  to  the 
questionnaire  based  only  upon  the  portion  of  tasks  you  were  able 
to  accomplish. 
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APPENDIX  B 


AUTOMATED  POSITION  DATA  ANALYSIS 
INTERVIEW  GUIDE 


Date 

Name  _ 

(First)  (M.I.)  (Last) 

Military  Rank  or  Civilian  Pay  Grade  _ 

MOS  or  civilian  Job  Series _ 

Time  as  MOS  Analyst  in  Months _ 

Number  of  MOS  Actions  Completed _ 
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NOTE  TOR  THE  EVRLTJRTZON  PARTICIPANTS 


Because  PDAT-JA  represents  a  prototype  system,  the 
developers  are  extremely  interested  in  the  input  of  MOS  analysts 
who  use  the  software  in  performing  position  data  analysis.  To 
assist  in  gathering  this  data,  the  developers  have  included  a 
pre-formatted  questionnaire  addressing  basic  topics  that  effect 
the  useability  and  effectiveness  of  the  software.  Once  you  have 
familiarized  yourself  with  PDAT-JA,  and  have  used  it  for  at  least 
four  weeks  to  analyze  position  data,  the  developers  will  return 
to  interview  you  and  complete  the  questionnaire.  In  the  mean 
time  however,  please  feel  free  to  use  this  questionnaire  as  a 
guide  while  performing  your  evaluation. 

Thank  you  in  advance  for  your  participation  in  the 
development  of  this  vital  personnel  proponent  tool. 

INSTRUCTIONS 

Responses  to  all  the  questions  in  this  questionnaire  should 
be  based  on  the  following  scenario.  If  the  scenario  does  not 
completely  fit  the  MOS(s)  you  are  performing  analysis  on,  please 
approximate  the  answers  based  on  your  experience.  Please  perform 
the  scenario  described  below  to  the  maximum  extent  possible.  You 
will  be  Interviewed  in  about  four  weeks  on  your  experiences  and 
the  questionnaire  completed  by  the  person  conducting  the 
interview. 

When  responding  to  questions  that  ask  you  to  rate  data 
accuracy  and  reliability,  please  rate  them  using  this  scale: 

1  -  Very  Low  Accuracy  and  Reliability 

2  -  Low  Accuracy  and  Reliability 

3  -  Less  Than  Average  Accuracy  and  Reliability 

4  -  Average  Accuracy  and  Reliability 

5  -  More  Than  Average  Accuracy  and  Reliability 

6  -  High  Accuracy  and  Reliability 

7  -  Extremely  High  Accuracy  and  Reliability 
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SECTION  1 

PDAT-JA  Asslstttd  Position  Data  Analysis 


1.  Application  of  ths  Army's  Avsrago  Grade  Distribution  Matrix 
(AGM) 

a.  Did  you  apply  AGM  to  assess  the  career  progression  and 
stability  of  the  MOS(s)  during  position  data  analysis? 

YES  _ 

NO  _ 

If  YES,  please  give  the  average  number  of  tim«is  _ 

If  NO,  please  explain:  _ 


b.  On  average,  how  long  (in  mins.)  did  it  take  you  to  apply 

the  AGM  to  an  MOS  using  PDAT-JA?  _ 

c.  Once  you  developed  an  SGA  for  a  merge  or  restructure 

action  and  applied  it  to  an  MOS(s)  averaging  5,000 
positions,  on  average,  how  long  did  the  computer  take  to 
perform  the  analysis  required  to  be  able  to  apply  the 
AGM  to  the  modified  structure?  _ 

d.  How  would  you  rate  the  accuracy  and  reliability  of  the 
AGM  data  provided  by  PDAT-JA? 

Circle  One:  12  34567 

e.  On  average,  how  many  times  did  you  modify  TAADS  and 

perform  analysis  of  modified  MOS  positions  to  attain  the 
proper  grade  structure?  _ 

2.  ASI  Data  Collection 

a.  Did  you  summarize  ASI  Authorizations  by  Grade  when 
performing  position  data  analysis? 

YES  _ 

NO 
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If  YES,  how  many  ASI(s)  were  associated  with  the  MOS(s) 
you  were  analyzing?  _ 

If  YES,  how  long  did  it  take  for  the  computer  to  perform 
analysis  necessary  to  gather  the  ASI  data?  _ 

If  NO,  please  explain:  _ 


b.  Did  you  also  perform  ASI  analysis  after  application  of  a 
new  or  revised  SGA? 

YES  _ 

NO  _ 

If  YES,  how  many  times?  _ 

c.  How  would  you  rate  the  accuracy  and  reliability  of  the 
ASI  data  provided  by  PDAT-JA? 

Circle  One:  1  2  3  4  5  6  7 

3.  SQI  Data  Collection 

a.  Did  you  summarize  SQI  Authorizations  by  Grade  when 
performing  position  data  analysis? 

YES  _ 

NO  _ 

If  YES,  how  many  SQI(s)  were  associated  with  the  MOS(s) 
you  were  analyzing?  _ 

If  YES,  how  long  did  it  take  for  the  computer  to  perform 
analysis  necessary  to  gather  the  SQI  data?  _ 

If  NO,  please  explain:  _ 
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b.  Did  you  also  perform  SQI  analysis  after  application  of  a 
new  or  revised  S6A? 

YES  _ 

NO  _ 

If  YES,  how  many  times?  _ 

c.  How  would  you  rate  the  accuracy  and  reliability  of  the 
SQI  data  provided  by  PDAT-JA? 

Circle  One:  1234567 

4.  NACOM  Data  collaotion 

a.  Did  you  sximmarize  Major  Army  Command  (MACOM)  MOS 
authorizations  by  grade? 

YES  _ 

NO  _ 

If  YES,  how  long  did  it  take  for  the  computer  to  perform 
the  analysis  necessary  to  gather  MACOM  data?  _ 

If  NO,  please  explain:  _ 


b.  Did  you  also  perform  MACOM  analysis  after  application  of 
a  new  or  revised  SGA? 

YES  _ 

NO  _ 

If  YES,  how  many  times?  _ 

c.  How  would  you  rate  the  accuracy  and  reliability  of  the 
MACOM  data  provided  by  PDAT-JA? 

Circle  One:  1234567 

5.  TOE  Versus  TDA  Data  Collection 

a.  Did  you  summarize  MOS  authorizations  TOE  versus  TDA  by 
grade? 

YES  _ 

NO 
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If  YES,  how  long  did  It  take  the  computer  to  perform  the 
analysis  necessary  to  gather  TOE  versus  TDA  data? 


If  NO,  please  explain: 


b.  Did  you  also  perform  TOE  versus  TDA  analysis  after 
application  of  a  new  or  revised  SGA? 

YES  _ 

NO  _ 

If  YES,  how  many  times?  _ 

c.  How  would  you  rate  the  accuracy  and  reliability  of  the 
TOE  versus  TDA  data  provided  by  PDAT-JA? 

Circle  One:  1234567 

CONUS  Versus  ocoNUS 

a.  Did  you  summarize  MOS  authorizations  CONUS  versus  OCONUS 
by  grade? 

YES  _ 

NO  _ 

If  YES,  how  long  did  it  take  the  computer  to  perform  the 
analysis  necessary  to  gather  CONUS  versus  OCONUS  data? 


If  NO,  please  explain: 


b.  Did  you  also  perform  CONUS  versus  OCONUS  analysis  after 
application  of  a  new  or  revised  SGA? 

YES  _ 

NO 
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If  YES,  how  many  tines?  _ 

c.  How  would  you  rate  the  accuracy  and  reliability  of  the 
CONUS  versus  OCONUS  data  provided  by  PDAT-JA? 

Circle  One:  1  2  3  4  5  6  7 

Modifications  to  TAADS  Document s 

a.  Did  you  make  changes  to  the  MOS(s)  data  on  the  automated 
TAADS  worksheet  based  on  new  or  revised  SGA? 

YES  _ 

NO  _ 

If  YES,  for  up  to  three  MOS(s)  containing  an  average  of 
5,000  positions,  how  long  did  it  normally  take  you  to 
accomplish  this  task?  _ 

If  NO,  please  explain:  _ 


b.  On  average,  how  many  times  did  you  perform  this  task? 


c.  How  would  you  rate  the  accuracy  and  reliability  of  the 
data  provided  by  PDAT-JA  once  you  entered  your 
modifications? 

Circle  One:  1234567 
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SECTION  2 

PDAT-JA  Usttrs  Evaluation  Quaationnaira 


1.  Input  and  Output  Davloaa 

a.  Did  the  keyboard  provide  you  with  the  speed  and  accuracy 
needed  to  Input  data  for  performing  position  data 
analysis? 

YES  _ 

NO  _ 

If  NO,  please  explain:  _ 


b.  Do  you  feel  that  another  type  of  control  such  as  a  mouse, 
would  be  better? 

YES  _ 

NO  _ 

If  YES,  what  type  and  why:  _ 


b.  Can  the  information  on  the  screen  displays  be  easily  read 
and  interpreted? 

YES  _ 

NO  _ 

If  NO,  what  causes  the  difficulty?  _ 


What  would  make  it  easier? 


2 .  Data  Display 

a.  Do  you  always  have  enough  information  on  the  screen  to 
perform  the  actions  required  in  each  step  of  analysis 
supported  by  PDAT-JA? 

YES  _ 

NO  _ 
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If  NO,  please  explain: 


b.  Are  there  ever  tines  when  the  information  presented  is 
more  than  you  used? 

YES  _ 

NO  _ 

If  YES,  when?  _ 

Does  this  interfere  with  performing  your  job?  Please 
explain: 


T 


c.  Is  the  organization  of  information  displayed  on  the 
screen  helpful  for  doing  your  job? 

YES  _ 

NO  _ 

If  NO,  what  would  you  change  and  why?  _ 


d.  Have  you  ever  found  information  coding  (highlighted 
displays)  to  be  a  problem? 

YES  _ 

NO  _ 
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If  YES,  When? 


e.  Do  the  charts  used  in  this  system  ever  confuse  you? 

YES  _ 

NO  _ 


If  YES,  which  ones?  Why? 


f.  Is  there  anything  that  you  would  like  see  changed  or 
added  to  software? 

YES 

NO  _ 

If  YES,  Please  explain:  _ 


3 .  User  Input 


a.  Have  you  ever  had  any  problems  entering  data  into  the 
MPCTAADS  Worksheet? 

YES  _ 

NO  _ 

If  YES,  when?  Please  explain:  _ 
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b.  Would  you  change  anything  about  the  way  the  system 
requires  you  to  enter  data? 

YES  _ 

NO  _ 

If  YES,  please  explain:  _ 


4.  Feedback  and  Error  Handling 

a.  Are  the  displayed  messages  such  as  the  ones  provided  when 
editing  a  worksheet,  ever  confusing  or  difficult  to 
remember? 

YES  _ 

NO  _ 

If  YES,  please  explain:  _ 


b.  Do  error  messages  give  you  enough  information  to  correct 
your  errors? 

YES  _ 

NO  _ 


If  NO,  please  explain: 


c.  Is  there  anything  about  the  way  the  system  handles  errors 
or  provides  feedback  that  you  would  change? 

Please  explain:  _ 


d.  Can  you  always  correct  data  rntry  errors  that  you  have 
made? 


YES _ 

NO  _ 

If  NO,  please  explain: 


e.  Can  you  always  return  to  your  place  in  the  program  after 
seeking  help  (pressing  the  Fl  function  key)  ? 


YES  _ 

NO  _ 

If  NO,  Please  explain: 


5 .  Interactive  Control 


a.  Are  the  menus  formatted  so  that  you  can  quickly  and 
easily  select  options? 

YES  _ 

NO  _ 
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If  NO,  please  explain 


b.  Can  you  always  retrieve  the  information  you  need  from  the 
computer  data  base? 

YES  _ 

NO  _ 

If  not,  please  explain:  _ 


c.  Have  you  ever  had  any  problems  moving  or  positioning  the 
cursor? 


YES  _ 

NO  _ 

If  YES,  when? 


6 .  Command  Methods 


a.  Are  the  instructions  you  must  provide  to  the  computer  to 
initiate  an  action  (Enter,  Cancel,  Escape,  -i-,  -,  etc.) 
confusing  or  difficult  to  use? 

YES  _ 

NO  _ 
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If  YES,  When? 


b.  Are  there  additional  computer  commands  that  would  assist 
you  in  performing  position  data  analysis? 


YES  _ 

NO  _ 

If  YES,  please  explain: 


1 


c.  Are  there  computer  commands  available  that  are  repetitive 
or  not  needed  to  accomplish  your  job? 


YES  _ 

NO  _ 

If  so,  what  are  they? 


7 .  General 


Are  there  any  other  comments  regarding  the  usability  of  the 
computer  interface  of  PDAT-JA  that  you  would  like  to  make? 

YES  _ 

NO 


B-14 


If  YES,  what  are  they? 


SBCTION  3 

Os«rs  Mmnual  Quastionnaira 

1.  Installation  and  Start-Up  Prooadures 

Do  the  procedures  provide  you  with  enough  information  to 
perform  install  the  software  and  begin  running  PDAT-JA? 

YES  _ 

NO  _ 

If  NO,  please  explain:  _ 


2 .  Jump  Start 


Does  the  "JUMP  START"  section  provide  you  with  enough 
Information  to  operate  the  system  and  enable  you  to  import 
TAADS  data,  perform  analysis,  develop  and  annotate 
worksheets,  and  produce  reports? 

YES  _ 

NO  _ 

If  NO,  please  explain:  _ 


3.  Summary  of  System  Concepts  and  Reports  Production 


a.  Do  the  system  concepts  provided  in  "  A  VIEW  FOR  THE 
ANALYST"  supply  enough  information  to  provide  a  basic 
knowledge  of  what  phases  of  position  data  analysis  PDAT- 
JA  is  designed  to  support? 

YES  _ 

NO  _ 
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If  NO,  please  explain 


b.  Does  the  section  also  provide  enough  Information  on  the 
types  and  purposes  for  the  reports  provided  by  PDAT-JA? 

YES  _ 

NO  _ 

If  NO,  Please  Explain:  _ 


4.  Description  of  System  Operation 


a.  Is  the  information  provided  in  the  "DETAILED  DESCRIPTION 
OF  OPERATION"  precise  enough  for  you  to  execute  all  the 
operations  and  functions  of  PDAT-JA? 

YES  _ 

NO  _ 

If  NO,  Please  explain:  _ 


b.  Are  there  places  where  the  information  presented  is  more 
than  you  need  to  perform  the  task? 


YES 

NO 
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If  YES,  please  explain 


c.  Is  the  organization  of  the  material  presented  in  the 
manual  helpful  for  doing  your  job? 

YES  _ 

NO  _ 

If  NO,  please  explain:  _ 
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APPENDIX  C 


PDAT-JA  BASELINE  EVALUATION 
INTERVIEW  GUIDE 


Date 

Name  _ 

(First)  (M.I.)  (Last) 

Military  Rank  or  Civilian  Pay  Grade  _ 

MOS  or  Civilian  Job  Series _ 

Time  as  MOS  Analyst  in  Months _ 

Number  of  MOS  Actions  Completed _ 


C-1 


EVALUATION  OT  POSITION  DATA  ANALYSIS  PERFORMED  IN  A  MANUAL  MODE 

Answer  to  all  questions  should  be  based  on  the  following 
scenario.  If  the  scenario  does  not  fit  any  MOS  action  you  have 
previously  performed  analysis  on,  please  approximate  your  answers 
based  on  your  experience. 

When  responding  to  questions  that  ask  you  to  rate  data 
accuracy  and  reliability,  please  rate  them  using  this  scale: 

1  -  Very  Low  Accuracy  and  Reliability 

2  -  Low  Accuracy  and  Reliability 

3  -  Less  Than  Average  Accuracy  and  Reliability 

4  -  Average  Accuracy  and  Reliability 

5  -  More  Than  Average  Accuracy  and  Reliability 

6  -  High  Accuracy  and  Reliability 

7  -  Extremely  High  Accuracy  and  Reliability 

SCENARIO; 

You  are  performing  position  data  analysis  on  an  MOS 
restructure  or  merger  action  which  contains  5,000  authorized 
positions.  You  will  first  analyze  TAADS  data  base  on  the  current 
force  structure.  Then  you  will  make  modifications  to  the  force 
structure  on  TAADS  and  perform  position  data  analysis  to 
ascertain  what  changes  occurred  as  a  result  of  your 
modifications.  You  are  to  respond  to  the  questions  based  only  on 
the  steps  performed  in  position  data  analysis.  Time  or  effort 
spent  in  performance  of  SGA  development,  personnel  data  analysis, 
physical  demands  analysis,  etc.,  are  not  to  be  considered  in  your 
response . 


BASELINE  EVALUATION 
Position  Data  Analysis 


1.  Application  of  tbs  Army's  Avarags  Grads  Distribution  Matrix 
(AGM) 

a.  Do  you  apply  AGM  to  assess  the  career  progression  and 
stability  of  the  MOS(s)  during  position  data  analysis? 

YES  _ 

NO  _ 

If  YES,  please  give  the  average  number  of  times  _ 

If  NO,  please  explain:  _ 


b.  On  average,  how  long  (in  mins.)  does  it  take  you  to 

apply  the  AGM  to  an  MOS?  _ 

c.  Once  you  have  developed  an  SGA  and  applied  it  to  one  or 

more  MOS(s)  totaling  5,000  positions,  on  average,  how 
long  does  it  take  you  to  perform  the  grade  structure 
counts  required  to  be  able  to  apply  the  AGM  to  the 
modified  structure?  _ 

d.  How  wculd  you  rate  the  accuracy  and  reliability  of  the 
data  obtained  by  manual  counts? 

Circle  One:  1  2  3  4  5  6  7 

d.  On  average,  how  many  tiroes  must  you  modify  TAADS  and 
perform  counts  of  modified  MOS  positions  to  attain  the 
proper  grade  structure?  _ 

2.  ASI  Data  Collection 

a.  Do  you  summarize  ASI  Authorizations  by  Grade  when 
performing  position  data  analysis? 

YES  _ 

NO  _ 
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b.  If  YES,  how  many  ASI(s)  are  associated  (on  average)  with 

the  N0S(8)  In  your  CMF?  _ 

c.  If  YES,  how  long  does  It  take  for  you  to  perform  the 

counts  necessary  to  gather  the  AS I  data?  _ 

d.  If  NO,  please  explain:  _ 


e.  Do  you  also  perform  these  counts  after  application  of  a 
new  or  revised  SGA? 

YES  _ 

NO  _ 

f.  How  would  you  rate  the  accuracy  and  reliability  of  the 
AS I  data  obtained  by  manual  counts? 

Circle  One:  1234567 

3.  SQI  Data  Collection 

a.  Do  you  summarize  SQI  Authorizations  by  Grade  when 
performing  position  data  analysis? 

YES  _ 

NO  _ 

b.  If  YES,  how  many  SQI(s)  are  associated  (on  average)  with 

the  MOS(s)  in  your  CMF?  _ 

c.  If  YES,  how  long  does  it  take  for  you  to  perform  the 

counts  necessary  to  gather  the  SQI  data?  _ 

d.  If  NO,  please  explain:  _ 
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e.  Do  you  also  perform  these  counts  after  application  of  a 
new  or  revised  S6A? 

YES  _ 

NO  _ 

f.  How  would  you  rate  the  accuracy  and  reliability  of  the 
SQI  data  obtained  by  manual  counts? 

Circle  One:  1234567 

MACOM  Data  Colleotion 

a.  Do  you  summarize  Major  Army  Command  (MACOM)  MOS 
authorizations  by  grade? 

YES  _ 

NO  _ 

b.  If  YES,  how  long  does  it  take  for  you  to  perform  the 

counts  necessary  to  gather  MACOM  data?  _ 

c.  If  NO,  please  explain:  _ 


d.  Do  you  also  perform  the  counts  after  application  of  a 
new  or  revised  SGA? 

YES  _ 

NO  _ 

e.  How  would  you  rate  the  accuracy  and  reliability  of  the 
MACOM  data  obtained  by  manual  counts? 

Circle  One:  1234567 

TOE  Versus  TDA  Data  Collection 

a.  Do  you  summarize  MOS  authorizations  TOE  versus  TDA  by 
grade? 

YES  _ 

NO  _ 

b.  If  YES,  how  long  does  it  take  for  you  to  perform  the 

counts  necessary  to  gather  TOE  versus  TDA  data?  _ 
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c.  If  NO,  please  explain: 


d.  Do  you  also  perform  the  counts  after  application  of  a 
new  or  revised  SGA? 

YES  _ 

NO  _ 

e.  How  would  you  rate  the  accuracy  and  reliability  of  the 
TOE  versus  TDA  data  obtained  by  manual  counts? 

Circle  One:  1234567 

6.  CONUS  versus  ocoNUS 

a.  Do  you  summarize  MOS  authorizations  CONUS  versus  OCONUS 
by  grade? 

YES  _ 

NO  _ 

b.  If  YES,  how  long  does  it  take  for  you  to  perform  the 
counts  necessary  to  gather  CONUS  versus  OCONUS  data? 


c.  If  NO,  please  explain: 


d.  Do  you  also  perform  the  counts  after  application  of  a 
new  or  revised  SGA? 

YES  _ 

NO  _ 

e.  How  would  you  rate  the  accuracy  and  reliability  of  the 
CONUS  versus  OCONUS  data  obtained  by  manual  counts? 

Circle  One:  1234567 
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Nodi  float  ions  to  TAAD8  Doouaants 


a.  Do  you  make  pen  and  ink  changes  to  MOS  data  on  TAADS 
based  on  new  or  revised  S6A? 

YES  _ 

NO  _ 

b.  If  YES,  for  up  to  three  MOS(s)  containing  an  average  of 

5,000  positions,  how  long  does  it  normally  take  you  to 
accomplish  this  task?  _ 

c.  If  NO,  please  explain:  _ 


d.  On  average,  how  many  times  must  you  perform  this  task? 

e.  How  would  you  rate  the  accuracy  and  reliability  of  the 
manually  modified  data? 

Circle  One:  1  2  3  4  5  6  7 
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